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At a time when emphasis is being placed on quality, standards, and accountability the 
research world is spending more time assessing the value and impact of library organizations. 
The paper argues that research in this area requires sociological and psychological research 
skills. Qualitative data are as important as statistics. Different research methodologies are 
identified in a review of some recent studies. The result of such work must be made known, 
and new research is required to explore the relationship between research and the decisions 
taken by professionals and policy makers. We must ensure that value and impact studies have 
value and impact. 
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Paper , 

At a time when so much emphasis is being placed on quality, standards, and accountability, it 
should not come as a surprise that the research world is spending more time assessing the 
value and impact of organizations. Serious difficulties arise however when researchers try to 
deal with these issues using inappropriate tools. Tools that can be used to assess outputs, may 
not be the same as those we use to evaluate outcomes. Value and impact studies are much 
more than mere measurement. Statistics are just a small part of the reality of the library and 
any meaningful evaluation has to go beyond simplistic quantification. Even the quality guru 
Deming, who is often associated with the use, if not over use of statistics, has admitted that the 
most important things in life can not be measured. How does one measure love, kindness, and 
generosity? Many of the measures that are set out in the managerialist literature are often 
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substitutes for qualities that are intangible or indirect.. 

Academic librarians have observed that, "relatively little attention has been paid to qualitative 
measures, or to output measures, but indicators tvhich fail to take such factors into account 
will be inadequate and misleading." (Joint Funding Councils' 1993) In the public sector it has 
been argued that, "performance can seldom be expressed in a meaningful way by quantitative 
data only. To a great extent, analysis of performance has to be based on qualitative 
descriptions and statements." (AiVidsson 1986) If library services are to be evaluated in terms 
of their impact on individuals, and society as a whole, then we require sociological and 
psychological research skills. It means that we must be concerned with the "soft" as well as 
"hard" areas. As Stewart and Ranson(1988) have observed: "Performance monitoring in the 
public domain is not merely concerned with effectiveness in achieving stated values, but with 
unexpected impact, and of values denied." 

In this brief review I am therefore going to concentrate on qualitative methodologies as used in 
some recent studies. We can also borrow research methodologies from other fields. For 
example, although a great amount of work has been carried out on "the library user", we still 
know relatively little about the functions of library materials in terms of their impact on the 
individual or the community. In a Sheffield study, on the value and impact of imaginative 
literature, we plan to combine the methodology, developed in our previous social audit work, 
with the "uses and gratification's" techniques used by researchers in the mass media (e.g. 
Brown 1976, McQuail 1969), and the reader response approach which is seen as integral to the 
understanding of literature. Similarly, in evaluating and planning services for communities, 
there are techniques such as cognitive mapping, which may help library managers gain a 
clearer impression of people's perception of an area or locality. 

Perceptions play a significant part in value and impact studies, and research from other areas 
points to the dangers of making management, or indeed political, decisions on the basis of 
users' perceptions alone. Simply asking users is often not enough. Users know what they like, 
they sometimes know what information they need, but they often do not know what is possible 
for the library to provide. Some explanation of the different perceptions between librarians and 
users can be found in the literature on the psychology of prediction. For instance Slovic et al 
(1982) Tversky and Kahneman (1982) suggest that people judge events as likely if instances of 
it are easy to imagine or recall. What people can recall will depend on their knowledge and 
experience of the area. Library users by definition will have personal experience of using 
libraries. They will therefore have some relevant knowledge but as Stewart and Walsh (1989) 
suggest, from a public service perspective, in a number of areas this knowledge will be 
incomplete. 

The public library service is above all about providing equality of access to information, ideas, 
and works of imagination, and it is important that this is reflected in the way services are 
evaluated. With equity, as with anything else, performance has to be evaluated against 
objectives. In the UK there are national standards, such as those set by the Audit Commission, 
but the evaluation of equality is still in its infancy. There is a tendency to rely to a great extent 
on numbers. For example counting the number of books in a foreign language, or the number 
of people from a particular community, or background, who use, or do not use a service. These 
kinds of data are, of course, important, if nothing else they are an indicator of an authorities' 
commitment. However such figures tend only to deal with inputs and outputs. In evaluating 
equality, people's experience of using the service, and their perceptions of it, can be used to 
evaluate outcomes. 



ERIC 



Social Impact 

Such qualitative outcome indicators are often a more meaningful way of assessing the value 
and impact of a service and its achievements. This, more qualitative approach, has been used 
in a number of recent studies of social impact. In Beyond book issues Matarasso (1998) 
reviewed library based projects entered for the Library Association Community Initiative 
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Award, and assessed the extent to which they have produced social benefits and seeks to 
identify factors which lead to success. Another piece of work, undertaken by the Community 
Development Foundation, looked at working with local people to develop indicators of social 
benefit. (Harris 1998) 

The Sheffield Social Audit study was the subject of a paper at last year's IFLA (Usherwood & 
Linley 1999) so suffice it to say that the technique has much in common with quality audits, as 
defined by Percy-Smith (1992), and implemented by a number of local authorities with regard 
to recreation, transport and information technology services. It is has also been used by a 
variety of voluntary organizations, and companies such as Traidcraft and The Body Shop. To 
quote Gray (1995) social accounting has experienced "a (long overdue) resurgence as 
academics ...look for new ways of providing accounts of organisational life." The technique 
can be used to examine the success or failure of a portfolio of activities and services offered by 
a particular kind of library. For example inner city, urban or rural branch libraries or those 
serving designated areas of poverty. 

Another Sheffield project, on XhQ Impact of library closures and reductions in opening hours 
(Proctor, Lee & Reilly 1999), involved before and after studies of public library users in 
communities where closures and reductions were taking place. The data show that when a 
local library closes up to a third of adults may be deprived of access to the public library. 

Worst affected are young children and elderly people. Parents and teachers told the researchers 
that the impact of library closure was devastating. Although some children were reading as 
much as before, their choice and quality of reading suffered dramatically. People felt that only 
a local library helped them to feel a part of their own community and play an active part in it. 

It helped them to be part of a local information "network", reduced loneliness and, encouraged 
fiiendships. 

The Sheffield strike research is an example of opportunistic research. In the Summer of 1995 
Sheffield Libraries closed down for eight weeks. This gave researchers the chance to find out 
how people responded to the lack of a library service. Over 500 library users were interviewed 
after the libraries re-opened. Other local libraries and bookshops were surveyed to see whether 
people had transferred their use there. A survey of telephone callers using the information 
services was also carried out. The research showed that the social value of the local public 
library has been underestimated. People enjoy the experience of going to the public library, 
whether or not they need to borrow books or seek information. It seems to make a significant 
contribution to their quality of life. This seems especially true in communities with a high 
incidence of economic and educational deprivation. The local library can be an important 
resource for personal development, particularly when users have had a poor experience of 
formal education. The data also demonstrated the extent to which reading is an essential and 
critical factor in the lives of library users. Most people cannot afford to buy enough books to 
replace those provided through the library service. Turning from the established to the 
electronic technology, colleagues at Manchester Metropolitan University are currently looking 
at the value and impact of End-User IT services in public libraries. They are using what they 
describe as a "mix of methodologies, including desk analyses, structured interviews, and 
observation. Another Sheffield project seeks to demonstrate the value and impact of public 
libraries on educational attainment and the reduction of educational disadvantage in British 
Society. It will investigate the role of the public library in supporting and encouraging 
learning, in its widest sense, among the educationally disadvantaged. Learning will be taken to 
include both the absorption of new knowledge and information, and the acquisition of 
information seeking and retrieval skills. 

The value and impact of library research. 

Crucially the results of research must be made known. For a number of years we have strongly 
argued the need to establish a centre for the effective dissemination of public library research. 
We emphasised that this should not be confined to work carried out in academic institutions 
and other research centres but also include the "hidden research" that is undertaken as part of 
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the day to day management of libraries. We have made some suggestions regarding the 
dissemination of research of interest to the library community. These include : Establishing a 
clearing house for the dissemination of reports and surveys prepared by different library 
organizations; developing a data base of these activities and; establishing a web site which 
would keep practitioners, academics, policy makers and others up to date with research being 
carried in library authorities, in universities, and other organizations with an interest in the 
field. 

New research is required to explore the relationship between the decisions taken by library 
managers and policy makers, and the research that is undertaken by universities and other 
research organizations. In suggesting this approach I do of course recognise that decisions are 
constantly made in libraries which are not always based on systematic research. Such has been 
the pace of change that there will be times when, "Life cannot wait for social research to catch 
up." (Shipman 1988). Equally the quality of decisions may be improved if more people within 
library organizations catch up with research that has been undertaken. 

Particular attention should paid to the ways in which research results are communicated within 
library organizations and how far the research focus of the organization is at risk through 
dependency on individuals rather than systems. Key professional staff and policy makers need 
to recognize the value of research as a contributor to performance. As researchers we need to 
ask, how far library services can be described as having a research ethos, through which the 
value of research is conveyed to professionals and policy makers. In the early 1970s a number 
of research officer posts were established in library directorates but most seem to have 
disappeared, although I did notice recently the appointment of a Research & Support Officer 
in a British public library. 

The 'implementation' of research findings involves more than simply applying a recipe. 
Research is often about innovation and all innovations require organizational change to some 
degree if they are to be implemented effectively. For research to be 'implemented', someone 
needs to know about it, they must influence the organization and persuade policy makers that 
change along the indicated lines will be valuable. "Thus whilst knowledge of the results of 
research may be important, it is rarely by itself sufficient to change practice, and other factors 
need to be t^en into account" (Watt 1996). Senior library managers need to persuade and 
cajole others in their organizations to try out new things, and to invest in people and other 
resources to provide the training that changes will require. In other words, implementing 
research findings is a major organizational task and not something simply to be bolted on. 

Much of the recent work on the dissemination and implementation of research is to be found in 
the clinical and medical literature. Some of the approaches used in the Health Services GRIPP 
(Getting Research into practising and purchasing) initiative and by Canadian clinicians (Clark 
1995), could be adopted for use in the library sector. We can also note that many of the models 
used by health researchers derive from the social sciences. For example in their use of Rogers 
(1995) work on the diffusion of innovations. Whatever model we adopt we must ensure that 
our research; illuminates the activities of library users; helps professionals and policy makers 
assess the impact of the services they provide, and enables them to identify the reasons for 
their success or failure. The research should help managers guide and monitor the service, and 
improve the way the value of the service is reported to policy makers. It should enable 
stakeholders to make better judgements about the service, and affect organizational behaviour. 
In short we must make sure that value and impact studies have value and impact. 
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